This invention relates to a video security system physically located at a site being protected; and, more particularly, to an essential interface between one or more cameras positioned about the site for monitoring purposes, and an alarm unit, as well as the interface between a camera and a remote operator. Whereas there is a growing number of video based security systems, a site control unit (SCU) of the present invention for a particular video based security system is unique. The SCU locally controls a plurality of cameras or other imaging equipment, processes perceived images using algorithms which identify potential sources of false alarms as well as detects the presence of an intruder, provides an alarm if the detected intruder is of a class for which an alarm is to be sounded, and provides video, audio and data output information to a control center, in accordance with a predetermined set of operating conditions.
Conventional security systems protect a building by employing a series of make/break contacts strategically placed at doors, windows, and other potential entry points. When a contact is broken and an alarm is sounded or relayed back to a control station located within the building, nearby the building, or remotely to a central control station of the security company employed to protect the premises from which the company monitors many premises. In addition to the make/break sensors, security companies may also use P.I.R. (passive infra red) sensors which sense heat differences caused by animate objects such as humans or animals, vibration sensors which, when placed upon a window for example, detect when the window is broken, and microwave sensors. As with the make/break sensors, when any one of the sensors indicates a detection, a system alarm is indicated. The alarm is also given if the alarm unit within the building to which the sensors are connected senses that the path to a sensor is interrupted or broken.
With current alarm systems, all that the receiver of an alarm, whether at a local or remote central station, knows is that an alarm has occurred. The system operator has no other knowledge by which he can determine if the alarm signals the presence of a real intruder, or if it is a false alarm. Sensors are notorious for going off during inclement weather (they are sensitive to large electromagnetic fields such as will occur during lightning conditions, etc.). Certain sensors are triggered by the presence of such animals as cats within their precincts. Regardless of why they occur, all false alarms detrimentally effect the efficiency and operation of a security system.
There are many criteria which determine whether or not an alarm condition exists. For example, when a person opens a door monitored by a sensor, a potential alarm condition is created. However, an alarm system typically has a keypad or other coded system control by which, if an appropriate entry is made, signifies that the alarm condition is not to be acted upon. Rather, the entrant is someone authorized to enter the premises. Further, the class of intruder (human, animal, etc.) may be perfectly acceptable in one set of circumstances, but not so in another. The common situation is one where if an intruder is human, that situation should result in an alarm being given. However, if the intruder is a cat a dog, for example, giving an alarm is inappropriate and such an alarm would be a false alarm. In an aviary, on the other hand, the presence of a human might be perfectly acceptable, but the presence of a cat or dog, should result in an alarm being given. Or, in an area where food is stored, the presence of a human may again be perfectly acceptable, but the appearance of mice or rats should trigger an alarm. As discussed hereafter, there is currently no alarm system which can be programmed to classify intruders, discriminate between acceptable and non-acceptable classes of intruders, and provide an appropriate alarm when the presence of an intruder from a non-acceptable class is detected.
False alarms plague the security system industry. While the situation is annoying when a false alarm is relayed to a local monitoring station, it becomes even worse when the alarm is relayed 2,000 miles or more to a security company's remote central station. Here, operators must use their experience of the particular circumstances surrounding the alarm (i.e., local weather conditions, past occurrences at that particular site, etc.), in order to make a determination as to whether or not the alarm is real. If their knowledge and experience tells them the alarm signifies an actual intrusion, they must then relay the alarm to the local police for that site so the police can conduct a further investigation.
There are numerous examples of when an alarm either did not work, or was rendered ineffective, simply because an operator at a control station had no insight into the facility where the alarm system was installed and from which an alarm emanated. In one anecdotal example, an alarm was triggered by a cat left in a residence. The police were called but discovered nothing, not even the cat, because the cat hid from strangers. After this situation repeated itself over a period of several hours, the police finally refused to investigate further. From that point on, the residence was essentially not protected. Over the years, situations resulting from false alarms have continued to worsen. Now, police often require some confirmation or evidence of an intrusion before investigating, or else they will give priority to those situations where they have greater certainty an intrusion has occurred. Security system companies have addressed this issue by providing an audio (or "listening-in") capability to the system. This enables the monitor to hear actual movement on the premises, the sound of voices, glass breaking, cabinets or drawers being opened, etc., with this information also being relayed to the authorities. Furthermore, in many locales, if the authorities investigate the report of an alarm and discover nothing, they will send the security company requesting the investigation a bill for their services.
In response to this situation, the security industry has begun to extensively use video cameras to constantly monitor premises. Use of cameras solves the problem of not just reacting to a make/break contact. The shortcomings with camera surveillance is that one needs to have a continuously connected communication channel with the sensor (camera), and the operator at the local or remote console must continuously monitor the video. Some systems have attempted to combine video with another sensing mechanism, I.R., for example, so that actuation of the video is controlled by the other sensor first sensing the presence of an intruder. For, if video is continuously required for a properly functioning system, a communications channel must be connected between the site and the monitoring station from the time the alarm system is energized. Because a monitoring period often exceeds 12 hours, the communication costs are high. To further control costs, the cameras employed at the monitored site are often slow scan cameras whose output is compressed onto POTS (plain old telephone system) lines (typically using 28.8k modems) with transmission rates of 1 frame of video over a 1-5 second interval. At the receiving end, the operator now must deal with two issues. First, because the frame rate is slow, what the operator sees is not what is necessarily occurring at that moment. Second, and more importantly, most of the time the operator will see nothing out of the ordinary. Yet, the operator must maintain a constant vigilance. This is a serious problem because it has been estimated that after watching a security system camera observing an unchanging scene for as little as 5 minutes, an operator's performance diminishes rapidly to the point where the operator is essentially ineffective after 30 minutes. One result of this, of course, is that false alarms still occur. As a consequence, the only real advantage video monitoring offers is that should an intrusion occur and should the operator notice it, then the relayed information sent to the local police will get high priority because of the certainty of the situation. Apart from this distinct advantage, the deficiencies of such a system are that it is very labor intensive, operator efficiency is usually very low, and communications costs are very high.
To overcome these problems while still providing the alarm system operator live images of an intrusion is the subject of the present invention. For, it is now possible, using the SCU described herein, to relay definitive information to the local police of an intrusion, as well as capture, maintain, and transmit images of the intrusion to the police or other authorities.
In co-pending U.S. patent application Ser. No. 08/771,991 (now abandoned) and U.S. Pat. Nos. 5,870,471, 5,937,092, and 5,956,424; the teachings of which are incorporated herein by reference, there is described a system and method for i) continuously viewing a scene to detect the presence of an intruder with a very low probability of false alarms and with a high probability of detection; and ii) a method for authenticating an image, and relaying the authenticated image from the protected site to a remote, viewing site. The fundamental detection process described in these applications involves establishing a reference scene (reference image) and comparing an image from the present scene (current image) with that reference. It is then determined whether any differences exist between the present and reference images. If the contents of the two images markedly differ, the result is interpreted as an intrusion of some kind within the scene having occurred. The detection process includes comparing, on a pixel by pixel basis, the current image with the reference image to obtain a difference image. In accordance with the process, any non-zero pixel in the difference image indicates the possible presence of an intrusion, after image artifacts such as noise, aliasing of the video, and movement within the scene not attributable to a life form (animal or human) such as the hands of a clock, screen savers on computers, oscillating fans, etc., have been accounted for. Because the system and method use an absolute difference technique with pixel by pixel subtraction, the process, as described in U.S. Pat. No. 5,937,092 is sensitive to surface differences between the scene but insensitive to light-on-dark or dark-on-light changes, and thus is very sensitive to any intrusion within the scene. Furthermore, each pixel represents a gray level measure of the scene intensity that is reflected from that part of the scene. Gray level intensity can change for a variety of reasons. The most important of these is a new physical presence at that particular part of the scene. The ability to make this determination, in accordance with the teachings set forth in these co-pending applications, removes from the human operator of the alarm system the initial responsibility of determining whether an intrusion results from a new human presence or otherwise. This, in turn, eliminates the need for the human operator to continuously monitor all of the cameras on the premises of the site being protected. Also, since monitoring is performed by the SCU, there is no need for a continuous communication path between the protected site and a remote operator. As described hereinafter, upon detection of an intrusion, a communication path is established, and high frame rate and high quality video is transferred from the site to the operator. This enables the operator to concur with the SCU's evaluation of an intrusion. Further, during an intrusion, and as described hereinafter, high resolution samples ("snapshots") of the video are taken by the SCU for later transfer to the alarm system operator at the operator's location. These samples are transferred using lossless compression techniques and are authenticated so as to be later admitted into court for prosecution purposes. Therefore, the same function has been achieved with a continuous relaying of the video remotely using slow scan cameras and with a high operator load.
Some efforts have previously been made to incorporate the recognition of objects, including humans, whose presence is detected or sensed in an image, into some type of control unit. For example, U.S. Pat. No. 5,305,390 to Frey et al., teaches recognition of persons or objects by height as they pass through a doorway or entrance. The intrinsic sensor is an active laser beam, and the system of Frey et al. operates by measuring the height of an object passing through an aperture (doorway) to classify the object as a person or not. Therefore, the system is a height discriminator rather than an object recognition or classification system. Thus, for example, if a person crawls through the aperture, they will probably be designated as a non-human.
U.S. Pat. No. 5,289,275 to Ishii et al., is directed to a surveillance monitoring system using image processing for monitoring fires and thefts. The patent teaches use of a color camera for monitoring fires and a method of comparing the color ratio at each pixel in an image to estimate the radiant energy represented by that pixel. A resulting ratio is compared to a threshold with the presence of a fire being indicated if the threshold is surpassed. A similar technique for detecting the presence of humans is also described. The patent teaches the use of image processing together with a camera to detect the presence of fires and abnormal objects.
U.S. Pat. No. 4,697,077 to Yausa et al. also teaches use of a camera to detect the presence of an object. Once an anomaly is detected because of differences in the comparison of an original and a later image, the system automatically dials and sends a difference image, provided the differences are large enough, to a remote site over a telephone e line. At the remote site, the image is viewed by a human. While teaching some aspects of detection, Yausa et al. does not go beyond the detection process to attempt and use image processing to recognize that the anomaly is caused by a human presence.
U.S. Pat. No. 4,257,063 which is directed to a video monitoring system and method, teaches that a video line from a camera can be compared to the same video line viewed at an earlier time to detect the presence of a human. However, here, the detection device is not a whole image device, nor does it make any compensation for light changes, nor does it teach attempting to automatically recognize the contents of an image as being derived from a human. Similarly, U.S. Pat. No. 4,161,750 teaches that changes in the average value of a video line can be used to detect the presence of an anomalous object. Whereas the implementation is different from the '063 patent, the teaching is basically the same.
All of these previous attempts at recognition have certain drawbacks, whether the type of imaging, method of processing, etc., which would result in either an alarm not being provided when one should, or in false alarms being given. The system and method of the present invention overcome these problems or shortcomings to reliably provide accurate indications of human intrusion in an area being monitored by a security system. Such an approach is particularly cost efficient because it reduces the necessity of guards having to patrol secured areas (which means each area will be observed only on an infrequent basis unless there are a large number of guards), while ensuring that any intrusion in any area is not only observed, but an appropriate alarm is sounded in the event of an intrusion by an intruder of an appropriately designated class for which an alarm is to be given.
The above examples deal only with recognition. As important for a sight control unit for a facility is the ability to handle multiple cameras, to perform recognition, timely transmit critical information, images, audio, and data to the facility monitor, authenticate images so transmitted, and operate interactively with the facility monitor or system operator.